Internet text retrieval systems accept a statement for requested information in terms of a search query S made up of a plurality of keywords T1, T2, . . . Ti, . . . Tn and return a list of documents that contain matches for the search query terms. To facilitate the performance of such searches on internet databases, search engines have been developed that provide a query interface to the information containing sources and return search results ranked sequentially on how well the listed documents match the search query. The effectiveness in obtaining desired results varies from search engine to search engine. This is particularly true in measurement of effectiveness in searching certain product support databases which can be heavily weighted with technical content. This information can be in a number of natural languages, both analog and digital form, and in a number of different formats, and in multiple machine languages. Further, while a search engine may be effective in identifying and ranking subject matter in one technical area, it can be ineffective in selecting the most pertinent references in another technical area. Therefore it would be desirable to be able to automatically evaluate the effectiveness of search engines in turning up and ranking documents in selected technical areas.
Therefore it is an object of the present invention to provide a tool for automatic measurement of the ability of a search engine to provide comprehensible search results.
Another object of the present invention is to automatically compare the ability of search engines to provide desired research results.
It is further an object of the present invention to measure a search engines ability to provide requested information to searchers in a selected technical area.